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CONFERENCE ON AUTOMATED DECISION-MAKING AND PROBLEM SOLVING 
THE THIRD DAY: ISSUES DISCUSSED 


Walter W. Hankins, Jack E. Pennington, and L. Keith Barker 


INTRODUCTION 


On May 19, 20, and 21 of 1980, Langley Research Center hosted at Langley, 
a Conference on Automated Decision-Making and Problem Solving. The purpose of 
the workshop was to explore related topics in Artificial Intelligence, Opera- 
tions Research, and Advanced Control Theory; and, in particular, to assess 
existing techniques, determine trends of development, and identify potential 
for application in automation technology programs within NASA. The first two 
days consisted of formal presentations by university representatives ' from each 
of the three disciplines in open sessions. A list of the papers presented is 
shown in table 1. The third day's session included the invited speakers and 
a small number of NASA representatives. Discussion centered on current techno- 
logy in automation and how NASA can and should interface with the university 
community in a mutually beneficial manner to advance this technology. This 
paper summarizes the discussion from the third day proceedings. Proceedings 
of the first two days will be published separately. Editorial comment, implicit 
in statements made in this paper, are the authors' interpretation of what they 
believe to have been the viewpoint of the person making the comments or the 
viewpoint of the group as a whole. In the following section, the intent has 
been to objectively summarize the issues raised during discussions. 


ROUND-TABLE DISCUSSIONS 


On the third and final day of Langley's 1980 Conference on Automated 
Problem Solving and Decision-Making, the consultants, who had presented formal 
papers during open sessions of the previous two days, along with NASA 
representatives and a small number of invited guests, engaged in informal 
round-table discussions. It was expected that these discussions would summarize 
the important issues put before the conference and clarify points of agreement 
and disagreement on them. In addition, these discussions were expected to 
provide insight and guidance to NASA program managers and researchers in 
developing future NASA automation technology programs. Participants in these 
discussions are shown in table 2. 

Jack Pennington opened the session with brief background remarks regarding 
NASA's interests in automation technology and, in particular, Langley's 
interests which led to the convening of this conference. He pointed out that 
Langley is presently accumulating information from universities, industry, and 
various government agencies to be used in determining the state-of-the-art in 
automated problem solving and decision-making. This task is an initial step 
in the present NASA Automation Technology Program, which calls for a phased 
build-up of an intelligent systems technology effort, beginning with the 





decision process and later adding perception and ultimately effector control. 
Pennington proposed that perhaps the greatest benefit to current Langley 
interests would come from the panel formulating an effective structure or 
mechanism which could be used to define the state-of-the-art in automated 
problem solving and decision making. 

Dr. William Gevarter from NASA Headquarters strongly endorsed Pennington* s 
proposal and expounded upon it, stating that an attempt had already been made 
to devise a structure for doing this. It would break down the work being done 
in the three fields of Artificial Intelligence, Operations Research, and Modern 
Control Theory. Groupings might consist of examples such as **A11 technologies 
for real-time control'* or "All processes that can design a plan", etc. The 
breakdown could include other such information as the researcher doing the work, 
the processing rate of the technique, and expected completion date of current 
research. Gevarter suggested that possibly a device like a super Venn diagram 
could be constructed to organize and contain this information. He expressed 
belief that much commonality among the methods and techniques, used by the three 
disciplines represented, would surface from completing the suggested groupings. 
He also expressed the hope that some deeper structure for the decision process 
might be found as well. In addition, Dr. Gevarter asked the panel to discuss 
the boundaries among their disciplines. 

From this point on, the specific panel member expressing a particular idea 
or point of view will not usually be identified because this summary is pre- 
liminary and has not been reviewed by individual panel members for their 
concurrence. 

Representatives of the Artificial Intelligence (AI) community addressed 
the issue of the state-of-the-art definition by stating that a structure, such 
as that suggested by Dr. Gevarter, could not very well be applied to Artificial 
Intelligence because AI is a preparadigm discipline without enough age in the 
subject to have a classification scheme or taxonomy of techniques. It was 
proposed that instead, AI might be portrayed through a series of case studies 
of running systems. Five to ten specific systems which are very good at 
specific applications and which are different enough to illustrate different 
approaches to problem solving might be selected. Expert systems such as 
Stanford's "Mycin" for diagnosing infectious diseases, or speech understanding 
systems such as Carnegie-Mellon' s "Heresay II", are examples of these. 

Although some insistence continued throughout the panel discussions that 
concentration on the categorizing of functions and techniques for automated 
problem solving continue, there was general agreement among the panel members 
that the process was not desirable nor likely to be fruitful. It was, conse- 
quently, not done. 

One panel member observed that from the presentations of the previous two 
days, the interests of the presenters seemed to fall into three categories. 

One group was interested purely in theoretical issues. A second group was 
interested in numerical issues with emphasis on real-time operation and coping 
with the combinatorial difficulties. The third group was interested purely in 
symbolic computing in which functions like optimization are really not factors 
at all. He suggested that the latter two groups could probably benefit from 
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an exchange of ideas which would bring about a greater use of numbers by those 
interested in symbol manipulation, and vice versa. 

An attempt was made by several panel members to compare and contrast the 
fields of AI/ Control Theory, and Operations Research (OR) from the point of 
view of the nature of the problem area to which each is applicable and the way 
in which all three may interact in a complementary fashion to compose a complex 
problem-solving system. Within a single system the three might be thought of 
as forming a hierarchical structure ranging in function from strategic to 
tactical. Operations Research bridges the middle ground from the top-level, 
knowledge-based decision making of AI to the actuator control at the lowest 
level. It was stated that control theory operates in a space where the struc- 
ture is well known, but has to deal with complicated constraints. Operations 
Research, on the other hand, deals with much simpler constraints, but functions 
in a space whose structure is more obscure. In AI there is little numerical 
space at all. It is, rather, an abstract space whose structure cannot be 
represented by geometric constraints or the like. It is constrained by 
knowledge, where knowledge is statements of facts. The relationships and 
interpretations of the three disciplines within a single system were illustrated 
loosely through the example of an autonomous robot with some repair task to 
perform. Deciding which tools to transport to the worksite is clearly the kind 
of qualitative reasoning which should be supplied by AI . Determining how to 
get to the worksite is on the boundary between OR and AI , and computing leg 
movements which provide stable locomotion is clearly a control function. 

The Venn diagram of figure 1 was sketched by one of the consultants to 
illustrate the areas of common interest among OR, Controls, and Artificial 
Intelligence. No objections were made to the diagram or to the areas of exper- 
tise each of the disciplines could supply to each of the others. The suggestion 
that the broad areas on the diagram be discussed in more detail was rejected as 
being an exercise in exchanging textbooks. 

There were some brief comments made about what the three disciplines can 
learn from each other. For instance, expertise from Operations Research may 
benefit Artificial Intelligence in managing and maintaining order in its 
research. Controls may be able to benefit Operations Research by giving it 
more knowledge and awareness of processes themselves (as opposed to purely 
managing them). Controls on the other hand need AI techniques to order, guide, 
and manage the processes being controlled. 

The one issue on which the panel agreed virtually without exception was 
that NASA could best support interdisciplinary basic research in automation by 
funding some focused project. Not only did the university consultants agree 
on this, but they did so with much enthusiasm. The issue was raised early in 
the session and was the dominant theme throughout the remainder of it. Several 
general characteristics that the focus should have, and advantages of specific 
projects for advancing basic scientific knowledge were discussed and agreed 
upon. The project should be exciting. It should be exciting enough to capture 
the imagination of researchers in a number of fields and broad enough to require 
their talents. It should be narrow enough to fill specific NASA needs and 
requirements while being broad enough to fill needs that would exist even if 
NASA were not involved. An example of a highly focused automation project 
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sponsored by NASA was the project to develop an autonomous Roving vehicle to 
explore the Martian surface. One of the problems with the Mars Rover project, 
however, was that it was too specific. Only those people who wanted to go to 
Mars were interested in it. It was so specific that there was in it the well- 
defined idea that- it could fail. The proposed focus should be aimed well into 
the future (10 to 15 years). It should have reasonable assurance of continuity 
and funding commensurate with its size and objectives. Finally, it should have 
a mechanism for pooling research expertise and results. 

Focusing on a specific problem will promote organization. Once lines are 
defined, people tend to organize themselves along those lines. The AI people 
particularly seem to want something to get themselves all moving in the same 
direction. A suggestion was made that RFP's on focused projects could be sent 
out even in the absence of funding. Another argument made for a focus is that 
science progresses from specific applications to generalizations. 

Dr. Gevarter pointed out that NASA has already chosen Automatic Sequence 
Generation in preparation for a planetary flyby as an initial focus in 
automation. At present hundreds of people require 1 to 1.5 years to complete 
the task manually. The work which is already in progress has found that the 
scheduling aspects of the problem are much more difficult than the planning. 

In addition, automated scheduling of ground support services for earth orbital 
missions is being addressed by Goddard Space Flight Center. Langley Research 
Center, in conjunction with the University of Illinois, is evaluating methods 
of error prevention and correction for automated systems. 

One of the projects suggested for a NASA focus received very enthusiastic 
response from the panel as a whole. Essentially, a space manufacturing facility 
to be in operation in about 15 years was proposed. The space station would 
contain around 20 different kinds of teleoperators along with many other kinds 
of manufacturing facilities. About 20 people would man the facility and would 
control it from a command station of some type. The structure would be very 
light and flexible, thus requiring complex systems to control and stabilize it. 

Comments concerning the ways in which their disciplines would like to 
participate in a focused NASA project were made by several panel members. 
Although most comments tended to relate to the candidate focus described in 
the previous paragraph some were more general. Operations Research expressed 
a particular interest in the design of the manufacturing facilities. Controls 
proposed to model the structure and structural dynamics of the space station 
in a way that will provide useful information to both the higher level reasoning 
and to the lower level processes which control unwanted oscillations. The 
information normally available to the higher level decision process is some- 
times needed by the lower level controls as well. For instance, lower level 
controls need to be made immediately aware that a Shuttle-type vehicle is 
arriving so that compensatory actions can be initiated. Artificial Intelligence 
expressed an interest in managing the very large data based associated with the 
space station and performing deductive procedures on those knowledge bases. 

The data base could also be used to do qualitative simulations of proposed 
problem solutions. Constructing better and more useful data bases will require 
understanding how people understand how gadgets are linked together. 
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Several times during these discussions the possible need for a center of 
excellence in information sciences and automation within NASA was brought up. 
Apparently, some feel that focal point for these activities within NASA would 
afford them a more obvious, direct, and knowledgeable contact for communicating 
their research concerns. Likewise, they feel that the establishment of such 
a center would overtly demonstrate the sincerity of NASA's interest and support 
of this research. There did not appear to be much general interest by the group 
as a whole in pursuing the subject; however, the possibility was brought up that 
NASA might put together strawman proposals for centers of excellence and send 
them out to universities and others, who would critique them and respond as to 
how the university could interface with the new center. A viewpoint was also 
expressed that NASA should have a distributed center of excellence. Dr. Gevarter 
told the panel that the issue has been raised at the highest levels within NASA 
but that it has not been resolved. 

Within NASA, basic research must compete with ongoing projects for funding. 
Because of this, it is difficult to obtain funds for new research and maintain 
funding continuity in current research. 

There appeared to be a consensus that the Department of Defense (DOD), and 
the Defense Advanced Research Projects Agency (DARPA) in particular, has a 
much better solution to this problem than does NASA. First, DOD accepts the 
concept that a small, but important part of its overall funding, should be for 
basic research. Basic research projects, then, compete with each other for 
this funding, not with development projects. A second desirable characteristic 
of DOD funding was attributed to DARPA for the type of projects it funds. Those 
projects tend to be specific and focused and at the same time be of broad, 
general interest to the scientific community. Examples of highly successful, 
DARPA funded projects were cited such as: the development of the ARPANET; the 

project to develop a system for understanding continuous, connected speech; and 
the development of the universal computer language, ADA. 

The Artificial Intelligence community heavily emphasized the importance of 
the programming language LISP in what they do and in being able to communicate 
with them. They cited several of its advantages. Artificial Intelligence 
researchers write computer programs which manipulate programs. LISP makes this 
easy for them. In contrast, it is almost impossible to write such programs in 
most other computer languages because they are organized differently and 
function differently from LISP. These other languages tend to be syntax 
intensive, which gives the code a more pleasing appearance as well as one 
closer to the usual way of representing the entities and functions being 
encoded. This characteristic, however, complicates greatly constructing 
programs which manipulate programs while being perhaps of trivial benefit to 
the programmer. LISP provides a uniform data language. That is, coded 
instructions are treated in just the same way as data are. The user is not 
disconnected from the running program and has the entire computer library 
available to him all the time. He can manipulate and change whatever he wants 
including the compiler itself. The user approaches his task from an uncommon 
point of view. Rather than constructing a working program, from the beginning, 
he constructs an experiment which he then manipulates. In addition to these 
advantages, the LISP environment was praised for being a community of users 
which enhances the productivity of its members through cooperative iteraction. 
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The question of establishing within NASA basic technical capabilities was 
raised. What mechanism could be used to effect such a transfer from the 
universities? How much information does NASA need and want? Is it interested 
in "cocktail-party" knowledge, expert knowledge, or somewhere in between? NASA 
representatives really did not answer these questions, except to concede that 
of the three disciplines only in Controls does NASA really possess considerable 
expertise. University representatives suggested an exchange of experts as a 
possible mechanism to solve the problem. 


CONCLUDING REMARKS 


Langley's 1980 Conference on Automated Decision-Making and Problem Solving 
produced a useful exchange of ideas among experts in Control Theory, Operations 
Research, and Artificial Intelligence. It gave NASA personnel a clearer pers- 
pective of how the academic community view's NASA's role in automation 
technology research. 

All three disciplines agreed that as automation is applied to even larger 
and more complex systems and is applied in a much greater degree of completeness 
than ever before, that more interdisciplinary expertise will be required. They 
agreed that a greater awareness by each discipline of what the other two 
disciplines are doing is needed and that this conference was a needed step in 
that direction. 

To determine its proper role in the future development of Automation 
Technology, NASA needs to know what automation technology has already been 
developed, what is being developed, and how NASA can acquire existing technology 
to use as a basis from which to move forward. 

Two objectives of the conference failed to be achieved. A mechanism by 
which NASA could assess the state-of-the-art in automated problem solving was 
not defined. Likewise, no taxonomy of techniques, procedures, or methods used 
by the three disciplines was made. 

However, as mentioned previously, NASA has a broad background in controls 
and is conducting and sponsoring research at or near the cutting edge of control 
system technology. Thus, the state-of-the-art in Control Theory by itself 
is fairly well known within NASA. Furthermore, one consultant volunteered to 
supply NASA with a review of the current capabilities in Operations Research. 
This review along with formal papers from the open sessions should provide a 
good start toward defining the state-of-the-art in Operations Research. 

A state-of-the-art definition in Artificial Intelligence will probably 
be more difficult to put together. While being a serious problem, this 
difficulty is nevertheless understandable when the nature of AI as a 
prescience discipline is recognized. 

Artificial Intelligence has not seemed to be well understood outside the 
confines of its own community of researchers. What it is and what its present 
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capabilities are have been obscure. Representatives of this community stated 
several important characteristics of the discipline which considerably clarified 
these questions. Among the more significant of these are: 

1. AI is qualitative. It is a preparadigm science. 

2. There is very little generality in approaches taken to 
problem solving; rather, approaches are problem-specific. 

3. AI operates on and within a domain very different from that which 
defines and constrains the physical sciences. Its domain is 
knowledge (statements of fact) rather than mathematics and physical 
laws . 

4. The primary tool, medium for communication, and material for 
constructing functioning systems is LISP and its derivative 
computer languages. 

One concept which seemed to pervade the conference was that automation 
is reaching the stage in which algorithms of a strictly deterministic form 
will no longer suffice. A hierarchical structure must evolve in which these 
deterministic forms of planning, housekeeping, and control become subject 
to a form of supervisory decision making which is human- 1 ike • (fuzzy, inexact, 
and nonrepeatable) . This higher level decision making will come from modeling 
human intelligence and ultimately understanding the underlying principles 
through which it operates. The increasing use of heuristics (emphasized in 
formal presentations) is a good example of using human- like decision processes. 

The academic community obviously feels strongly that NASA should define 
one or more specific projects through which to fund basic research. The 
completion of these projects and their applications should be aimed well into 
the future, should be assured of reasonable funding continuity, and should be 
broad enough to be applicable beyond NASA. 

From the perspective of the conference as a whole, the following are 
concluded: 

(1) The need for research in automation technology and the 

desirability of developing this expertise within NASA are 
recognized. 

(2) Artificial Intelligence is not yet a science; it is qualitative 
rather than quantitative. However, it is rapidly emerging as a 
powerful tool to address large, complex, unstructured problems. 

(3) Interdisciplinary communication and cooperations among 
Artificial Intelligence, Control Theory, and Operations Research 
have been limited, and the need to increase communication of 
technology between the fields was identified as the only way 

to solve the complex problems of large scale systems operating 
in space and on Earth. 

(4) Combining the three disciplines into a single system results 
in a hierarchical structure with Artificial Intelligence the 
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top level management. Control Theory the bottom level worker 
(first-line-supervisor), and Operations Research the middle 
management interface. 

(5) The link 'between automated systems and human factors was reinforced 
Mechanizing decision functions now performed by man will require 
extensive research into understanding human intelligence and 
implementing new methods of human interface with these machines. 

(6) The identified method for NASA to obtain expertise in automation 
technology is to initiate focused research resulting in the 
demonstration of automated problem solving capabilities, which 
would offer potential benefits to a broad range of complex problems 
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Title 


Author 


NASA Crosscut Studies and 
Applications 

A Framework for Automated 
Decision-Making and Problem Solving 

Intelligent Controls for Advanced 
Automated Processes 

Implications of Behavioral Decision 
Theory for Automatic Decision- 
Making and Problem Solving 

Problem Solving with Uncertain 
Knowledge 

Sequential Decision-Making and 
Stochastic Networks 

Distributed Problem Solving and 
Natural Language Understanding 
Models 

Application of A1 Problem 
Solving Techniques to Computer- 
Aided Design 

Systems Modeling Past, Present 
and Future as Viewed from a 
Network Modeling Perspective 

Recent Research in Network 
Problems with Applications 

Problem Solving in Distributed 
Systems 

Large Scale System Theory 
Decentralized Control 

Decentralized Control 


Dr. William Gevarter 
NASA 

Dr. Ewald Heer 

University of Southern California 

Dr. George N. Saridis 
Purdue University 

Dr. William R. Ferrell 
University of Arizona 


Dr. Bruce Buchanan 
Stanford University 

Dr. Salah Elmaghraby 

North Carolina State University 

Dr. Charles Rieger 
University of Maryland 


Dr. Gerald Sussman 

Massachusetts Institute of Technology 


Dr. Gary Whitehouse 
University of Central Florida 


Dr. Gerald L. Thompson 
Carnegie-Mellon University 

Dr. Richard F. Rashid 
Carnegie-Mellon University 

Dr. Robert Tenney 

Massachusetts Institute of Technology 

Dr. Jason L. Speyer 
University of Texas * 


Table 1. Papers Presented During Open Sessions 
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TABLE 2. ATTENDEES OF FINAL SESSION 


Name 


Organization 


Dr. Willard Anderson 
Dr. L. Keith Barker 
Dr. Bruce Buchanan 
Dr. Robert Chien 
Mr. Richard DesJardins 
Dr. Salah Elmaghraby 
Dr. William Ferrell 
Mr. Leonard Friedman 
Dr. William Gevarter 
Mr. Walter Hankins 
Dr. Ewald Heer 
Mr. Ronald Larsen 
Mr. Jim Long 
Mr. Jesse Maury 
Mr. Alfred Meintel 
Mr. Jack Pennington 
Mr. Lloyd Purves 
Dr. Richard Rashid 
Dr. Charles Rieger 
Dr. George Saridis 
Mr. A1 Schy 
Dr. Jason Speyer 
Dr. Gerald Sussman 
Dr. Robert Tenney 
Dr. Gerald Thompson 
Dr. Gary Whitehouse 
Dr. Leonard Yarbrough 


NASA, LaRC 
NASA, LaRC 
Stanford University 
University of Illinois 
NASA, GSFC 

North Carolina State University 
University of Arizona 
Jet Propulsion Laboratory 
NASA Headquarters 
NASA, LaRC 

University of Southern California 
NASA, GSFC 

Jet Propulsion Laboratory 

NASA, GSFC 

NASA, LaRC 

NASA, LaRC 

NASA, GSFC 

Carnegie-Mellon University 
University of Maryland 
Purdue University 
NASA, LaRC 
University of Texas 

Massachusetts Institute of Technology 
Massachusetts Institute of Technology 
Carnegie-Mellon University 
University of Central Florida 
NASA, MSFC 
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Figure 1. - Venn diagram of interdisciplinary issues and expertise. 
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